Analysis of coding principles in the olfactory system and their application in cheminformatics

Unser Geruchssinn vermittelt uns die Wahrnehmung der chemischen Welt. Im Laufe der Evolution haben sich in unserem olfaktorischen System Mechanismen entwickelt, die wahrscheinlich optimal auf die Erfüllung dieser Aufgabe angepasst sind. Die Analyse dieser Verarbeitungsstrategien verspricht Einblicke in effiziente Algorithmen für die Kodierung und Verarbeitung chemischer Information, deren Entwicklung und Anwendung dem Kern der Chemieinformatik entspricht. In dieser Arbeit nähern wir uns der Entschlüsselung dieser Mechanismen durch die rechnerische Modellierung von funktionellen Einheiten des olfaktorischen Systems. Hierbei verfolgten wir einen interdisziplinären Ansatz, der die Gebiete der Chemie, der Neurobiologie und des maschinellen Lernens mit einbezieht

Ten thousand times faster: Classifying multidimensional data on a spiking neuromorphic hardware system.

Discrimination of sensory inputs is a computational task that biological neuronal systems perform very efficiently. Assessing the principles in those systems is a promising approach to develop technical solutions for many problems, such as data classification. A particular problem here is to train a classifier in a supervised fashion to discriminate classes in multidimensional data. We implemented a network of spiking neurons that solves this task using a neuromorphic hardware system, that is, analog neuronal circuits on a silicon substrate. This system enables us to do high-performance computation in a biologically inspired way, with spiking neurons as computational units. In this contribution, we illustrate solutions to technical challenges that occur when implementing a classifier on neuromorphic hardware. 

The network topology of the insect olfactory system provides a well suited template for a neuronal architecture processing multidimensional data. In our classifier network, the value of each dimension of a data vector determines the rate of a stochastically generated spike train. The spike trains are fed into non-overlapping populations of neurons. Those populations project onto an association layer with winner-take-all properties representing the output of the classifier. During classifier training, the weights in this projection are adjusted according to a firing-rate based learning rule. 

The values in multidimensional data sets are typically real numbers, but neuronal firing rates are restricted to values between zero and some maximal value. Hence, the data must be transformed into a positive, bounded representation. We achieved this by representing each data point as a vector of distances to a number of points in data space (“virtual receptors” [1]). The representation by virtual receptors inevitably introduces correlation between input dimensions. We reduced this correlation using lateral inhibition in the first neuronal layer, leading to a significant increase in classifier performance. We found that decorrelation was most efficient when we scaled the inhibitory weights according to the correlation between the connected populations. 

We ran our classifier network on a neuromorphic hardware system that runs at ten thousand times the speed of biological neurons, thus suited for high performance computing [2]. However, the considerable variance of rate-response sensitivity across hardware neurons decreased classification performance. We therefore developed a calibration routine to counteract the neuronal variance.

References

[1] Schmuker, M. and Schneider, G. (2007). Processing and classification of chemical data inspired by insect olfaction. Proc. Natl. Acad. Sci. U S A 104, 20285-20289. 
[2] Brüderle, D., Bill, J., Kaplan, B., Kremkow, J., Meier, K., Müller, E. and Schemmel, J. (2010). Simulator-like exploration of cortical network architectures with a mixed-signal VLSi system. In Proc. of IEEE Intern. Symp. on Circuits and Systems (ISCAS), 2784–8787

Optimized Particle Swarm Optimization (OPSO) and its application to artificial neural network training

BACKGROUND: Particle Swarm Optimization (PSO) is an established method for parameter optimization. It represents a population-based adaptive optimization technique that is influenced by several "strategy parameters". Choosing reasonable parameter values for the PSO is crucial for its convergence behavior, and depends on the optimization task. We present a method for parameter meta-optimization based on PSO and its application to neural network training. The concept of the Optimized Particle Swarm Optimization (OPSO) is to optimize the free parameters of the PSO by having swarms within a swarm. We assessed the performance of the OPSO method on a set of five artificial fitness functions and compared it to the performance of two popular PSO implementations. RESULTS: Our results indicate that PSO performance can be improved if meta-optimized parameter sets are applied. In addition, we could improve optimization speed and quality on the other PSO methods in the majority of our experiments. We applied the OPSO method to neural network training with the aim to build a quantitative model for predicting blood-brain barrier permeation of small organic molecules. On average, training time decreased by a factor of four and two in comparison to the other PSO methods, respectively. By applying the OPSO method, a prediction model showing good correlation with training-, test- and validation data was obtained. CONCLUSION: Optimizing the free parameters of the PSO method can result in performance gain. The OPSO approach yields parameter combinations improving overall optimization performance. Its conceptual simplicity makes implementing the method a straightforward task

An unsupervised neuromorphic clustering algorithm

Brains perform complex tasks using a fraction of the power that would be required to do the same on a conventional computer. New neuromorphic hardware systems are now becoming widely available that are intended to emulate the more power efficient, highly parallel operation of brains. However, to use these systems in applications, we need “neuromorphic algorithms” that can run on them. Here we develop a spiking neural network model for neuromorphic hardware that uses spike timing-dependent plasticity and lateral inhibition to perform unsupervised clustering. With this model, time-invariant, rate-coded datasets can be mapped into a feature space with a specified resolution, i.e., number of clusters, using exclusively neuromorphic hardware. We developed and tested implementations on the SpiNNaker neuromorphic system and on GPUs using the GeNN framework. We show that our neuromorphic clustering algorithm achieves results comparable to those of conventional clustering algorithms such as self-organizing maps, neural gas or k-means clustering. We then combine it with a previously reported supervised neuromorphic classifier network to demonstrate its practical use as a neuromorphic preprocessing module

Exploiting plume structure to decode gas source distance using metal-oxide gas sensors

Estimating the distance of a gas source is important in many applications of chemical sensing, like e.g. environmental monitoring, or chemically-guided robot navigation. If an estimation of the gas concentration at the source is available, source proximity can be estimated from the time-averaged gas concentration at the sensing site. However, in turbulent environments, where fast concentration fluctuations dominate, comparably long measurements are required to obtain a reliable estimate. A lesser known feature that can be exploited for distance estimation in a turbulent environment lies in the relationship between source proximity and the temporal variance of the local gas concentration – the farther the source, the more intermittent are gas encounters. However, exploiting this feature requires measurement of changes in gas concentration on a comparably fast time scale, that have up to now only been achieved using photo-ionisation detectors. Here, we demonstrate that by appropriate signal processing, off-theshelf metal-oxide sensors are capable of extracting rapidly fluctuating features of gas plumes that strongly correlate with source distance. We show that with a straightforward analysis method it is possible to decode events of large, consistent changes in the measured signal, so-called ‘bouts’. The frequency of these bouts predicts the distance of a gas source in wind-tunnel experiments with good accuracy. In addition, we found that the variance of bout counts indicates cross-wind offset to the centreline of the gas plume. Our results offer an alternative approach to estimating gas source proximity that is largely independent of gas concentration, using off-the-shelf metal-oxide sensors. The analysis method we employ demands very few computational resources and is suitable for low-power microcontrollers

Rapid Recognition of Olfactory Scenes with a Portable MOx Sensor System using Hotplate Modulation

© 2022 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works. This is the accepted manuscript version of a conference paper which has been published in final form at https://doi.org/10.1109/ISOEN54820.2022.9789654A café, the metro, a supermarket, a book store - many locations of everyday life have a specific smell. Recognising such olfactory scenes could inform personal activity tracking, environmental monitoring, and assist robotic navigation. Yet it is unclear if current Metal-oxide (MOx) sensor technology is sensitive and specific enough to achieve this. Factors like sensor drift, and sensitivity to ambient humidity and temperature further complicate the recognition of olfactory scenes. Hotplate temperature modulation has been suggested as a method to counter these drawbacks. We present an electronic nose based on MEMS-MOx sensors that support rapid hotplate temperature modulation with a 150 ms period. We recorded different natural olfactory scenes in an urban context. A linear SVM was able to recognise four olfactory scenes in single hotplate cycles with near-perfect performance when trained and tested on the same day, and 73% accuracy when tested in the same locations on the next day. Gas sensor responses yielded higher recognition accuracy than humidity, temperature, and pressure, which were also partly-location specific. Our results indicate that hotplate modulation enables recognition of natural odor scenes across extended timespans. These findings encourage the use of MOx-sensors as rapid sensing devices in natural, uncontrolled environments

Sparse Coding with a Somato-Dendritic Rule

© 2020 Elsevier Ltd. All rights reserved. This manuscript is licensed under the Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International Licence http://creativecommons.org/licenses/by-nc-nd/4.0/.Cortical neurons are silent most of the time. This sparse activity is energy efficient, and the resulting neural code has favourable properties for associative learning. Most neural models of sparse coding use some form of homeostasis to ensure that each neuron fires infrequently. But homeostatic plasticity acting on a fast timescale may not be biologically plausible, and could lead to catastrophic forgetting in embodied agents that learn continuously. We set out to explore whether inhibitory plasticity could play that role instead, regulating both the population sparseness and the average firing rates. We put the idea to the test in a hybrid network where rate-based dendritic compartments integrate the feedforward input, while spiking somas compete through recurrent inhibition. A somato-dendritic learning rule allows somatic inhibition to modulate nonlinear Hebbian learning in the dendrites. Trained on MNIST digits and natural images, the network discovers independent components that form a sparse encoding of the input and support linear decoding. These findings con-firm that intrinsic plasticity is not strictly required for regulating sparseness: inhibitory plasticity can have the same effect, although that mechanism comes with its own stability-plasticity dilemma. Going beyond point neuron models, the network illustrates how a learning rule can make use of dendrites and compartmentalised inputs; it also suggests a functional interpretation for clustered somatic inhibition in cortical neurons.Peer reviewe

Predicting voluntary movements from motor cortical activity with neuromorphic hardware

This document is the Accepted Manuscript version of the following article: A. Lungu, A. Riehle, M. P. Nawrot and M. Schmuker, "Predicting voluntary movements from motor cortical activity with neuromorphic hardware," in IBM Journal of Research and Development, Vol. 61, no. 2/3, pp. 5:1-5:12, March-May 1 2017. The version of record is available online at doi: 10.1147/JRD.2017.2656063. © 2017 by International Business Machines Corporation. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.Neurons in the mammalian motor cortices encode physical parameters of voluntary movements during planning and execution of a motor task. Brain-machine interfaces can decode limb movements from the activity of these neurons in real time. The future goal is to control prosthetic devices in severely paralyzed patients or to restore communication if the ability to speak or make gestures is lost. Here, we implemented a spiking neural network that decodes movement intentions from individual neuronal activity recorded in the motor cortex of a monkey. The network runs on neuromorphic hardware and performs its computations in a purely spike-based fashion. It incorporates an insect-brain-inspired, three-layer architecture with 176 neurons. Cortical signals are filtered using lateral inhibition, and the network is trained in a supervised fashion to predict two opposing directions of the monkey’s arm reaching movement before the movement is carried out. Our network operates on the actual spikes that have been emitted by motor cortical neurons, without the need to construct intermediate non-spiking representations. Using a pseudo-population of 12 manually-selected neurons, it reliably predicts the movement direction with an accuracy of 89.32 % on unseen data after only 100 training trials. Our results provide a proof of concept for the first-time use of a neuromorphic device for decoding movement intentions.Peer reviewe

Classifying chemical sensor data using GPU-accelerated bio-mimetic neuronal networks based on the insect olfactory system

No description supplie